Papers with dense retrieval models
Aggretriever: A Simple Approach to Aggregate Textual Representations for Robust Dense Passage Retrieval (2023.tacl-1)
Copied to clipboard
| Challenge: | Pre-trained language models have been successful in knowledge-intensive tasks, but recent research calls into question the robustness of these singlevector models. |
| Approach: | They propose to exploit knowledge in a pre-trained language model for dense passage retrieval by aggregating contextualized token embeddings into a dense vector. |
| Outcome: | The proposed model significantly improves the effectiveness of dense retrieval models on in-domain and zero-shot evaluations without introducing substantial training overhead. |
Questions Are All You Need to Train a Dense Passage Retriever (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing methods for dense retrieval require large supervised datasets with custom hard-negative mining and denoising of positive examples. |
| Approach: | They propose a new corpus-level autoencoding approach for training dense retrieval models that does not require labeled training data. |
| Outcome: | The proposed method matches or surpasses strong supervised performance levels on multiple QA benchmarks with no labeled training data or task-specific losses. |
GNN-encoder: Learning a Dual-encoder Architecture via Graph Neural Networks for Dense Passage Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to perform large-scale query-passage retrieval are term-based, but they lose interaction between query-pastage pairs. |
| Approach: | They propose to fuse query (passage) information into query representations via graph neural networks that are constructed by queries and their top retrieved passages. |
| Outcome: | The proposed model outperforms existing models on MSMARCO, Natural Questions and TriviaQA datasets and achieves the new state-of-the-art on these datasets. |
Breaking Boundaries in Retrieval Systems: Unsupervised Domain Adaptation with Denoise-Finetuning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing domain adaptation methods for dense retrieval models use unadapted rerank models, leading to imprecise labels. |
| Approach: | They propose to adapt a rerank model to the target domain before using it for label generation. |
| Outcome: | The proposed model achieves better results across three retrieval datasets. |
CRAFT: Training-Free Cascaded Retrieval for Tabular QA (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for open-domain table question answering require retraining or fine-tuning on new datasets. |
| Approach: | They propose a zero-shot, cascaded retrieval approach that uses a sparse retrieval model to filter a subset of candidates before applying more expensive dense models as re-rankers. |
| Outcome: | The proposed method outperforms state-of-the-art retrieval models on the NQ-Tables dataset. |
Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering (2023.acl-short)
Copied to clipboard
| Challenge: | Existing dense retrieval models are parameter-inefficient and underperform sparse counterparts. |
| Approach: | They propose a task-aware specialization for dEnse Retrieval architecture that enables parameter sharing by interleaving shared and specialized blocks in a single encoder. |
| Outcome: | The proposed architecture surpasses BM25 on questions and passages using 60% of the parameters as bi-encoder dense retrievers. |
Less is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak Decoder (2021.emnlp-main)
Copied to clipboard
Shuqi Lu, Di He, Chenyan Xiong, Guolin Ke, Waleed Malik, Zhicheng Dou, Paul Bennett, Tie-Yan Liu, Arnold Overwijk
| Challenge: | Dense retrieval requires high-quality text sequence embeddings to support effective search in the representation space. |
| Approach: | They propose a self-learning method that pre-trains the autoencoder using a weak decoder to push the encoder to provide better sequence representations. |
| Outcome: | The proposed model significantly boosts the effectiveness and few-shot ability of dense retrieval models on web search, news recommendation, and open domain question answering. |
Boosted Dense Retriever (2022.naacl-main)
Copied to clipboard
| Challenge: | DrBoost is a dense retrieval ensemble that is trained in stages to correct retrieval mistakes . it produces representations which are 4x more compact, while delivering comparable retrieval results. |
| Approach: | They propose a dense retrieval ensemble inspired by boosting that is trained in stages . they produce representations which are 4x more compact, while delivering comparable retrieval results . |
| Outcome: | The proposed model performs surprisingly well under approximate search with coarse quantization, reducing latency and bandwidth needs by another 4x. |
Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval (2024.naacl-long)
Copied to clipboard
| Challenge: | et al., 2020: performance of dense retrieval models in multilingual retrieval is limited due to uneven and scarce training data available across multiple languages. |
| Approach: | They propose a synthetic retrieval training dataset containing 33 languages for fine-tuning multilingual retrievers without human supervision. |
| Outcome: | The proposed model outperforms human-supervised retrieval models on three retrieval benchmarks. |
Evidence Retrieval for Fact Verification using Multi-stage Reranking (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evidence retrieval methods are limited by single-stage evidence extraction. |
| Approach: | They propose to use a multi-stage reranking paradigm to enhance the fact verification process by increasing the recall of sentences by 7.85%, tables by 8.29% and cells by 3% compared to the current state-of-the-art. |
| Outcome: | The proposed system outperforms state-of-the-art models and achieves a 93.63% recall rate for Wikipedia pages. |
HIL: Hybrid Isotropy Learning for Zero-shot Performance in Dense retrieval (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in dense retrieval models have brought ColBERT to prominence in information retrieval, but it is underperforming in zero-shot tasks. |
| Approach: | They propose a Hybrid Isotropy Learning architecture that integrates isotropic and anisotropic representations to improve zero-shot retrieval performance. |
| Outcome: | The proposed model outperforms the baseline ColBERT model in BEIR benchmarks. |
Domain Adaptation for Dense Retrieval and Conversational Dense Retrieval through Self-Supervision by Meticulous Pseudo-Relevance Labeling (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent studies have shown that dense retrieval models generalize less well than interaction-based models on out-of-distribution data sets. |
| Approach: | They propose to combine query-generation approach with self-supervision approach in which pseudo-relevance labels are automatically generated on the target domain. |
| Outcome: | The proposed approach is based on a T5-3B model for pseudo-positive labeling and hard negatives on conversational dense retrieval models. |
Simple Entity-Centric Questions Challenge Dense Retrievers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Open-domain question answering has exploded in popularity due to the success of dense retrieval models. |
| Approach: | They construct a set of simple, entity-rich questions based on facts from Wikidata and test their models against supervised datasets. |
| Outcome: | The proposed model outperforms sparse retrieval methods on open-domain question answering datasets by a large margin. |
CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that dense retrieval models are better at obtaining query-informed representations. |
| Approach: | They propose a dual-encoder approach that computes latent representations of query and document independently, but inference replaces the real query with a generated one. |
| Outcome: | The proposed approach outperforms previous dense retrieval models on in-domain and out-of-domain datasets. |
Enhancing the Ranking Context of Dense Retrieval through Reciprocal Nearest Neighbors (2023.emnlp-main)
Copied to clipboard
| Challenge: | Sparse annotation poses persistent challenges to training dense retrieval models . despite potential future endeavors to extend annotation, issue of false negatives persists . |
| Approach: | They propose a method that smooths out the annotation of unlabeled relevant documents . they use reciprocal nearest neighbors to estimate relevance and rerank candidates . |
| Outcome: | The proposed method reduces the issue of false negatives in contrastive learning by reducing sparsity. |
DMRetriever: A Family of Models for Improved Text Retrieval in Disaster Management (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models fail to handle the varied search intents inherent to disaster management scenarios, resulting in inconsistent and unreliable performance. |
| Approach: | They propose a new series of dense retrieval models tailored for disaster management that train on a three-stage framework with unsupervised contrastive pre-training and difficulty-aware progressive instruction fine-tuning. |
| Outcome: | The proposed model outperforms baseline models by 13.3 times and 33 times over baselines with only 7.6% of their parameters. |
CODER: An efficient framework for improving retrieval through COntextual Document Embedding Reranking (2022.emnlp-main)
Copied to clipboard
| Challenge: | Contextual document embedding reranking is an efficient and efficient retrieval framework. |
| Approach: | They propose a highly efficient retrieval framework that uses contextual document embedding reranking to incorporate ranking context into training. |
| Outcome: | The proposed framework reduces the computational overhead of a first-stage method and can be used as stand-alone retrieval models. |
Noisy Pair Corrector for Dense Retrieval (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing dense retrieval models assume that query-document pairs are exactly matched, resulting in mismatched-pair noise. |
| Approach: | They propose a novel approach to train an effective model with mismatched-pair noise. |
| Outcome: | The proposed model performs well on natural question and triviaQA, code-search benchmarks and SO-DS. |
PairDistill: Pairwise Relevance Distillation for Dense Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in dense retrieval have demonstrated remarkable efficacy compared to traditional sparse retrieval methods. |
| Approach: | They propose to use pairwise relevance distillation to leverage pairwise reranking to enrich the training of dense retrieval models. |
| Outcome: | The proposed method outperforms existing methods and achieves state-of-the-art results on multiple benchmarks. |
RPDR: A Round-trip Prediction-Based Data Augmentation Framework for Long-Tail Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Long-tail question answering presents significant challenges for large language models due to limited ability to acquire and accurately recall less common knowledge. |
| Approach: | They propose a data augmentation framework that selects high-quality easy-to-learn training data to enhance dense retrieval models. |
| Outcome: | The proposed framework improves on two long-tail retrieval benchmarks, PopQA and EntityQuestion, and shows that it outperforms existing retrievers on extremely long-tailed questions. |
When Rules Learn: A Self-Evolving Agent for Legal Case Retrieval (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing dense retrieval methods have achieved notable progress, but their effectiveness in legal case retrieval remains limited. |
| Approach: | They propose a self-evolving framework for rule-driven query rewriting that enhances BM25 without any parameter training. |
| Outcome: | The proposed framework outperforms non-evolutionary baselines, including human-designed rules and greedy rule selection, especially when powered by a high-capacity core LLM. |